-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Added support for Pub/Sub mode in MultiDbClient #3722
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: vv-active-active-pipeline
Are you sure you want to change the base?
Conversation
… into vv-pubsub-support
… into vv-pubsub-support
… into vv-pubsub-support
) -> "PubSubWorkerThread": | ||
def callback(): | ||
return self._active_pubsub.run_in_thread( | ||
sleep_time, daemon=daemon, exception_handler=exception_handler, pubsub=pubsub |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to provide the pubsub reference here? If a failover has occurred, shouldn’t _active_pubsub already contain all the necessary information? By passing in this reference, don’t we risk holding onto a pubsub object from a failed instance and potentially trying to work with it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need currently active pub/sub reference to be able to execute any commands on it in any point of time (f.e subscribe to more channels if needed or unsubscribe), if MultiDbClient is in Pub/Sub mode. Also we keep it to be able to re-initialise new pub/sub during failover with the same configuration and re-subscribe to the same channels.
We always update reference when failover happens, so we do not risk to keep stale reference.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, and it handles the state of self._active_pubsub
for which we call run_in_thread
, but then a pubsub
object is also provided as an argument - I was asking about this one.
r_multi_db.spublish('test-channel', data) | ||
sleep(0.1) | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do we expect to have happened in the pubsub_thread? Would it be worth validating whether it completed successfully or failed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In pubsub_thread we're validating if we receive expected message from subscribed channel
Do you think we need to verify something else? Like counter to prove that it was called > 0 times?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe a check of the thread object results. The thread might have raised an exception due to this assert failure, but unless the results of the thread object are checked, we might never see that something has failed.
Pull Request check-list
Please make sure to review and check all of these items:
NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.